Sonification of the self vs. sonification of the other: Differences in the sonification of performed vs. observed simple hand movements

Existing works on interactive sonification of movements, i.e., the translation of human movement qualities from the physical to the auditory domain, usually adopt a predetermined approach: the way in which movement features modulate the characteristics of sound is fixed. In our work we want to go one step further and demonstrate that the user role can influence the tuning of the mapping between movement cues and sound parameters. Here, we aim to verify if and how the mapping changes when the user is either the performer or the observer of a series of body movements (tracing a square or an infinite shape with the hand in the air). We asked participants to tune movement sonification while they were directly performing the sonified movement vs. while watching another person performing the movement and listening to its sonification. Results show that the tuning of the sonification chosen by participants is influenced by three variables: role of the user (performer vs observer), movement quality (the amount of Smoothness and Directness in the movement), and physical parameters of the movements (velocity and acceleration). Performers focused more on the quality of their movement, while observers focused more on the sonic rendering, making it more expressive and more connected to low-level physical features

When Intelligent Machines Say No

It is easy to imagine frictionless scenarios where giving intelligence to the tools we use only results in positive additions to our creative experiences. But as soon as we venture further than systems of recommendation and give our machine agency to enact the suggestions they formulate, we must take into account a power equation that is not a zero sum game. In creative networks of humans and machines, performing an action can be seen as imposing a limit on the process as it unfolds. The tools we use today already impose limits on our process, and these very limitations can be conducive to creativity. Therefore, the question of how to handle limitations could be a useful way of framing our design process when developing the intelligent machines of the future.  QC 20160512</p

Crafting New Interfaces for Musical Expression

This thesis collects and contextualizes several projects involving artistically directed prototyping where new artifacts have been developed, in multi-disciplinary groups of practitioners, for use in performance contexts. These projects and their resulting publications have been team efforts, and therefore all papers have more than one author. In the introduction, a complementary perspective to that of the publications is offered, engaging with the characteristics of the digital innards of these artifacts and their digital material qualities. The stance that software source code is a design material is argued, and the notion of the crafting coder is used to view processes that use code as material for artistic creation. Code is also prominently featured in the introductory chapter with examples of some of the central components of the sound processing techniques that have been successfully used in the projects described in this thesis. The artifacts that are described in the thesis are: The Throat, an instrument for augmenting the singing voice using gestural control in real-time, The Vocal Chorder, a string based instrument using full-body interaction that also allows for audience participation through an installation mode, The Charged Room, a video tracking installation that lets users manipulate sound by moving across a stage, and Nebula, a garment that senses the users movements and responds with sound. These artifacts have been evaluated in the context they are designed for, and not only tested in laboratory settings, to make sure that the knowledge produced is valid. Several performances and peda-gogical courses have been used as empirical foundation for the claims of empowerment, expressivity, and performance qualities ascribed to the developed artifacts.  QC 20151202</p

When Intelligent Machines Say No

It is easy to imagine frictionless scenarios where giving intelligence to the tools we use only results in positive additions to our creative experiences. But as soon as we venture further than systems of recommendation and give our machine agency to enact the suggestions they formulate, we must take into account a power equation that is not a zero sum game. In creative networks of humans and machines, performing an action can be seen as imposing a limit on the process as it unfolds. The tools we use today already impose limits on our process, and these very limitations can be conducive to creativity. Therefore, the question of how to handle limitations could be a useful way of framing our design process when developing the intelligent machines of the future.  QC 20160512</p

Crafting New Interfaces for Musical Expression

This thesis collects and contextualizes several projects involving artistically directed prototyping where new artifacts have been developed, in multi-disciplinary groups of practitioners, for use in performance contexts. These projects and their resulting publications have been team efforts, and therefore all papers have more than one author. In the introduction, a complementary perspective to that of the publications is offered, engaging with the characteristics of the digital innards of these artifacts and their digital material qualities. The stance that software source code is a design material is argued, and the notion of the crafting coder is used to view processes that use code as material for artistic creation. Code is also prominently featured in the introductory chapter with examples of some of the central components of the sound processing techniques that have been successfully used in the projects described in this thesis. The artifacts that are described in the thesis are: The Throat, an instrument for augmenting the singing voice using gestural control in real-time, The Vocal Chorder, a string based instrument using full-body interaction that also allows for audience participation through an installation mode, The Charged Room, a video tracking installation that lets users manipulate sound by moving across a stage, and Nebula, a garment that senses the users movements and responds with sound. These artifacts have been evaluated in the context they are designed for, and not only tested in laboratory settings, to make sure that the knowledge produced is valid. Several performances and peda-gogical courses have been used as empirical foundation for the claims of empowerment, expressivity, and performance qualities ascribed to the developed artifacts.  QC 20151202</p

The throat III: disforming operatic voices through a novel interactive instrument

Practitioner-led artistic research, combined with interactive technologies, opens up new and unexplored design spaces. Here we focus on the creation of a tool for opera-singers to dynamically disform, change and accompany their voices. In an opera composed by one of the authors, the title-role singer needed to be able to alter his voice to express hawking, coughing, snuffling and other disturbing vocal qualities associated with the lead role Joseph Merrick, aka "The Elephant Man". In our designerly exploration, we were guided by artistic experiences from the opera tradition and affordances of the technology at hand. The resulting instrument, The Throat III, is a singer-operated artefact that embodies and extends particular notions of operatic singing techniques while at the same time creating accompaniment. It therefore becomes an emancipatory tool, putting a spotlight on some of the power hierarchies between singers, composers, conductors, and stage directors in the operatic world.Qc 20141117</p

New Scenic Subjects : Explorations of a System of Autonomous On-Stage Observers

QC 20160908</p

Sonification as Catalyst in Training Manual Wheelchair Operation for Sports and Everyday Life

In this paper, a study on sonification of manual wheelchair movements is presented. The aim was to contribute to both rehabilitation contexts and in wheelchair sports contexts, by providing meaningful auditory feedback for training of manual wheelchair operation. A mapping approach was used where key parameters of manual wheelchair maneuvering were directly mapped to different sound models. The system was evaluated with a qualitative approach in experiments. The results indicate that there is promise in utilizing sonification for training of manual wheelchair operation but that the approach of direct sonification, as opposed to sonification of the deviation from a predefined goal, was not fully successful. Participants reported that there was a clear connection between their wheelchair operation and the auditory feedback, which indicates the possibility of using the system in some, but not all, wheelchair training contexts.QC 20210913</p